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ABSTRACT 

The Job Literacy Project used the same adult literacy 
framework as the National Assessment for Educational Progress (NAEP) 
research and began the process of applying the framework to the world 
of work. The NAEP framework consisted of three scales: prose, 
document, and quantitative. The project tested a new methodology 
called Job Literacy Analysis, a modification of the job analysis 
procedures. The five jobs selected for study were entry-level 
positions in three clusters: service (food service worker, nurse 
assistant), administrative support (secretary, word processor), and 
mechanics, installers, repairers (data processing equipment 
repairer). Job Literacy Analysis resulted in a job literacy 
description, which included a representative collection of the most 
important materials required for competent performance on the job and 
the relevant (linked) tasks associated with those materials. Actual 
literacy materials used by job incumbents and information about how 
they are used were gathered during personal interviews. The materials 
were analyzed and reduced to a representative set of materials. The 
diaft job literacy description was reviewed by job experts and 
revised. A survey containing literacy materials, related tasks, and 
rating scales regarding use and importance of materials and tasks was 
distributed nationally to verify experts' judgments. The important 
literacy materials and related tasks were included in the final job 
literacy description. (Contains 26 references.) (YLB) 
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1. Introduction 

It is becoming increasingly clear why workplace literacy is an issue of great concern for 
employers today. Firms are finding that the problem they are facing is not one of selection, but 
instead one of a decreasing entry-level labor pool that is becoming more limited in its skill level. 
Evidence abounds indicating the severity of the situation. For example, in 1987 Nynex had to 
test 57,000 applicants in order to fill 2,100 positions for operators and repair technicians 
(Greenberg & Beilinson, 1989). The ratio of individuals tested to individuals hired was 27 to 1. 
At an advertising agency in Minneapolis the ratio of applicants to those qualifying was 20 to 1 
for secretaries and 10 to 1 for supply and mail clerks (Gold, 1989). 

The supply of new workers between 16 and 24 years of age is dwindling due to the 
decline in birthrates after 1960. And recent estimates indicate that about 27 million adults in 
the United States can be considered functionally illiterate, unable to read, write, calculate, or 
solve problems at a level that enables them to perform their job (Goddard, 1989; Gordon, 
Ponticell & Morgan, 1989). 

Specifically, Workforce 2000 . the well-known 1987 study from the Hudson Institute, 
indicated that 25 miUion workers need to improve their basic skills by 40 percent in order to 
meet the nation's work force requirements by the year 2000. The study also predicted that by 
the end of the century a bare 10 percent of the entry-level work force will be able to solve for "x" 
in a simple algebraic equation. And the National Assessment for Educational Progress (NAEP) 
studies that have been conducted by Educational Testing Service (ETS) (Kirsch & Jungeblut, 
1986) have shown that most young adults have basic literacy skills but are limited in their 
application of higher-level skills or strategies. Other recent studies have indicated that half of 
our country's industrial employees read at or below the eighth-grade level (Zemke, 1989). At 
the same time, literacy requirements of jobs are increasing. About 70 percent of the reading 
material sampled in a cross section of existing jobs is written between the 9th- and 12lh-grade 
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level (Mikulecky, 1982; Mikulecky, 1988; Rush, Moe & Storlie, 1986; and Sticht, 1982). The 
Hudson Institute reports that among the newest jobs emerging in the country, only 27 percent 
require lower literacy skill levels, while 41 percent require higher literacy skill levels (Johnston & 
Packer, 1987). 

Many other sources could be referenced to support these findings. It is clear, though, 
that the level of skiUs in the entry-level labor force is decreasing while job requirements are 
becoming more and more complex. 

For these reasons, employers are beginning to become more involved and to embrace 
the literacy problem confronting the United States and its employees. One recent article 
pointed out that "employee basic skills is here with us to stay as an issue, and we are going to 
have to devote thinking and resources to it from now on" (Jurmo, Wlggenhorn, Packer & 
Zeigler, 1989). In the last few years, training in basic skills has been on the rise. In 1986 
companies in the United States spent about $200 million on basic skills training; in 1989 they 
spent about $300 million (Carnevale & Gainer, 1990). One out of three American companies 
provides some form of basic skills training (Gold, 1989). In fact, a report published this year by 
the American Society for Training and Development revealed that 22 percent of major 
employers offer basic training in reading, 41 percent in writing, and 31 percent in arithmetic 
(1990). 

Although emphasis on literacy in the workplace dates back to the 1960s, when the 
American Right to Read effort stressed the critical role of literacy in rendering an individual 
employable, most major research projects in the United States have been undertaken only in the 
last few years (for a summary of these, see Norback & Rosenfeld, 1989; and Norback, Rosenfeld 
& Wilson, 1990). The notable exceptions, representing well-known research in the area broadly 
referred to as workplace literacy, are the work of Tom Sticht (e.g., Sticht, 1984, 1970, 1972, 1974, 
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1975, 1976, 1978), which focused on the military, and, more currently, the work of Larry 
Mikulecky and Irwin Kirsch, 

Sticht^s work encompassed many issues relating to the literacy level of individuals in 
various military occupational specialties (MOSs) in the Army. He researched and developed job 
reading task tests and studied their use in enhancing the literacy levels of Army personnel. He 
stressed the need for functional, job-related literacy training and found that adults learned job- 
related reading more quickly than general reading skills. Sticht recommended redesigning 
literacy materials used on the job and using audio materials in order to decrease the gap 
between the average readability level of publications used on the job and the reading ability of 
the individuals in that MOS or job, Mikulecky's research (Diehl & Mikulecky, 1980; Mikulecky, 
1982) has revealed that the nature of reading at school is different from the nature of on-the-job 
reading; e.g,, the strategies students use in reading may be less effective than those used by 
employees. Kirsch (e.g., Kirsch & Guthrie, 1984) has reported that the nature of reading on the 
job varies significantly across type of jobs, such as manager, technician or cler-cal worker. For 
example, on average, managers spend more on-the-job time reading society and science material 
than did the group as a whole; technicians did more reference reading at work than did the 
group as a whole, and clerical workers read more brief documents than did the group as a 
whole. It is beyond the scope of this report to describe the current research on workplace 
literacy in detail, except as it applies to the Job Literacy Project. 
II. The Contribution of NAEP Research 

A large contribution to the field has been made through the NAEP studies (Kirsch & 
Jungeblut, 1986) conducted by ETS, The NAEP framework conceptualizes literacy more 
broadly than ever before. The NAEP model consists of three scales that were designed to 
represent distinct and important aspects of literacy: prose literacy , which includes prose 



materials such as newspaper articles, magazines and books; document literacy ^ which covers the 
identification and use of materials found in tables, charts, forms, or indexes; and quantitative 
literacy , in which arithmetic operations are applied to information in printed form, for example, 
order forms, menus, or advertisements. 
III. The Job Literacy Project 

The Job Literacy Project described in this report broke new ground in a number of ways. 
It was conceptualized with the idea of beginning to apply the NAEP framework in a more 
focused way, i.e., to the workplace. One goal of the project was to collect and analyze literacy 
materials, i.e., materials that job incumbents read and use to get their work done. A second goal 
was to develop a new methodology to describe the literacy requirements of a variety of entry- 
level jobs. The approach is more focused because it centers on the job context rather than on 
the broader range of skills assessed in NAEP. It is also more focused because it is expected to 
be used in the design and development of diagnostic literacy assessment and training for 
individuals with limited literacy skills. 
IV. Background and Purpose 

The Job Literacy Project was funded by the ETS Office of Corporate Development. It 
had three major purposes: (1) to test a new methodology called Job Literacy Analysis (JLA), 
which is a modification of the job analysis procedures used successfully by ETS over the last 15 
years to focus on the literacy requirements of the job; (2) to describe the literacy requirements 
(materials and tasks) of the five entry-level jobs studied, and (3) to collect a broad variety of 
job-related materials (prose, document and quantitative materials actually used by job 
incumbents) for use in ETS literacy efforts. The results of the project were expected lo be used 
in the design and development of job-specific or industry-specific testing and training. 



The ETS Job Literacy Project was undertaken partly tu provide information that other 
current and past research projects in workplace literacy had not provided. One goal of the 
project was to ensure thai* the results could apply across settings, such as business, health care 
and government. To accomplish this, a cross section of job incumbents in the same job were 
interviewed, and a sizeable number of job incumbents (25 to 35 per job) were interviewed. The 
total number of job incumbents interviewed for the project was 190, a favorable comparison with 
the handful of job incumbents that were interviewed in the other major workplace literacy 
projects underway in the United States. 

In the body of this report, the selection of the jobs to be studied will be described and 
the basic steps of the methodology will be covered. A general description of the results of the 
methodology will be included and, finally, conclusions and recommendations will be discussed. 
V. Selecting the Jobs to be Studied 

The jobs selected for study were entry-level positions that were among the highest- 
volume, fastest-growing jobs from 1986 to 2000, according to Department of Labor (Bureau of 
Labor Statistics, 1987) projections. The occupations were representative of three different 
clusters defined by the Bureau of Labor Statistics. The jobs were also those which were 
expected to contain examples of prose, document, and quantitative literacy. The final criterion 
for the jobs to be analyzed in the project was overlap with ETS projects currently underway. 
The occupations chosen for study were: 

Service Cluster : 

1. food service worker 

2. nurse assistant 
Administrative Support Cluster : 

3. secretary 
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4. word processor 
Mechanics, Instaliers. Repairers Cluster : 

5. data processing equipment repairer 

Data were also gathered on two higher-level jobs: cook (the position into which a food service 
worker would be promoted) and higher-level or executive secretary. Analysis of these data was 
expected to occur in future projects. 

VI, The Job Literacy Analysis Methodology 

The new Job Literacy Analysis methodology tested during this project results in a job 
literacy description, which includes a representative collection of the most important materials 
required for competent performance of the job, and the relevant (linked) tasks associated with 
those materials. In the procedure, actual literacy materials used by job incumbents and 
information about how they are used are gathered during personal interviews. The materials 
are then analyzed and reduced to a representative set of materials. The resulting draft job 
literacy description is reviewed by a committee of job experts, including supervisors, and 
sometimes recruiters and trainers. The job literacy description is revised according to the 
committee's recommendations and then a survey is developed and distributed nationally to 
verify the judgments of the group of experts. The survey contains literacy materials, related 
tasks, and rating scales regarding the use and importance of the materials and tasks. Data 
analyses performed on the survey responses reveal any difference in importance of a material 
and its related tasks by gender or work setting. The important literacy materials and related 
tasks are included in the final job literacy description. 

VIL Results 

The results of the Job Literacy Analyses conducted will be summarized here. 
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Food Service Worker 

A total of 33 food service workers were interviewed across a variety of settings. 
Through content analysis and review by an Advisory Committee, the original 20 food service 
worker materials were reduced to 16, which were included in the job literacy description. 

Examples of the materials and tasks included in the description follow: 



Material: Sandwich lists 

Related Tasks: h Count or weigh food portions or read scale 

2. Label meat or cheese after slicing and wrapping 

3. Mark price code or label on wrapped sandwiches 



Recipes 

1. Read and follow directions to prepare items 

2. Modify recipe if needed (to accommodate more or fewer people) 

3. Perform taste tests 

4. Label meat or cheese after slicing and wrapping 

The job literacy description was distributed in survey format to approximately 800 food 
service managers across the United States. The managers were asked for information regarding 
the use and importance of the literacy materials and related tasks. 

Seventy percent of the managers surveyed returned a completed questionnaire. All 
materials except one (the Special Diet Tag or Plan) were rated as being used by approximately 
50 percent or more of survey respondents. The materials ranged in use from 46.5 percent for 
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the Hazardous Material Sheet to 96.8 percent for the Food Scale. The Special Diet Tag o Plan 
was used by only 18 percent of the respondents overall. However, 90 percent of the respondents 
in the health care setting used the materials and rated the tasks for which the materials were 
used as being extremely important. On this basis it was decided that the Special Diet Tag or 
Plan should be included in the domain of most important literacy materials used by a food 
service worker on his or her job. Therefore, the survey confirmed the expert opinion of the 
Advisory Committee regarding the job literacy description. 
Nurse Assistant 

A total of 29 nurse assistants, from hospitals, nursing homes, and rehabilitation centers, 
were interv'iewed and 99 nurse assistant literacy materials were obtained. The materials were 
reduced to a representative collection of the 30 most important literacy materials and related 
tasks through analysis and review by an Advisory Committee, The following are examples of the 
materials and t^sks included in the job literac>' description: 



Material: Hospital Incident Reports 

Related Tasks: 1. Read patient's hospital identification card and run through 

addressograph machine/enter ID information manually on forms 

2. Fill out form when patient accident occurs 

3. Obtain appropriate staff signatures, personally sign/witness forms 



Material: Intake and Output Record Sheets 

Related Tasks: 1. Review/fill out patient's daily information sheet, audio tapes, or 
assignment sheets from previous shift or for ne\t shift 
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2. 



Measure fluid intake and output 



3. 



Read patient's hospital identification card and run through 



addressograph machine/enter ID information manually on forms 



4. 



Record patient's intake and output 



c 



Calculate shift and daily totals for each patient on intake and 



output sheets 



Word Processor 

Nineteen word processors in a variety of settings were interviewed for the project and 
provided 121 literacy materials. The 121 word processor materials were analyzed and the 
materials and relevant tasks were compared to the secretary materials and tasks. Because of the 
overlap indicated by the comparison, the word processor materials were dropped from 
consideration as a separate category for the job literacy description and were combined with the 
secretary analysis to result in the "secretary /word processor" job literacy description. 



Sixty-three entry-level and higher-level secretaries were interviewed across settings and 
562 literacy materials were collected. The materials were then split into entry-level and higher- 
level by using the following criteria: current job title, job title of next-higher position, job title of 
next-lower position, job tasks performed, amount of experience, and specific tasks related to 
individual interview materials. After this process was completed, 22 of the 63 secretaries were 
classified as entry-level. 

The materials were reduced to a representative collection of the 35 most important 
literacy materials and related tasks through analysis and review by an Advisory Committee. 



Secretary 
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These materials and their related tasks comprised the job literacy description. Examples are 
shown below: 

Material: Computer and Other Instructions, Manuals Help Screens 

Related Tasks: 1. Format letters, tables, articles, and other materials 

2. Layout/create/alter charts, graphs, tables, and other materials 

3. Print out hard copy of letter, memo, or document 

4. Send maiJ/distribute to appropriate department/parties, internal 
or external 

5. Read instructions from computer menu/screen (e.g., available 
options, help screens) 

6. Read instructional manuals (e.g., word processing or other 
software package) 

7. Use standard reference materials (e.g., dictionaries, directories, 
schedules, catalogs, lists) 

8. Call up a program and issue commands to retrieve/update data or 
generate a report (data retrieval programs) 

9. Maintain existing files (e.g., file papers in the correct folders, 
electronically file) 

Material: Telephone Messages/Logs 

Related Tasks: 1. Send mail/distribute to appropriate department/parties, internal 
or external 
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2. Process forms/reports (e.g., fill out/update, attach documentation, 
distribute) 

3. Answer phones: direct, screen calls, write down messages, 
substitute for switchboard as needed 

4. Maintain logs (e.g., mail log, log of phone calls) 



Conclusions and Recommendations 

The goals of the Job Literacy Project were achieved: first, the project used the same 
Adult Literacy framework as NAEP (prose, document, and quantitative scales) and began the 
process of applying the framework to the world of work. Second, a new methodology was 
developed and successfully tested, as shown by the fact that the Advisory Committees reviewing 
the materials and tasks presented to them indicated that the materials were relevant and added 
relatively few materials. Also, when a survey was distributed, the majority of respondents 
indicated that materials were used on the job and the tasks for which they were used were 
judged to be important. Third, the literacy requirements (materials and tasks) of the entry-level 
jobs studied were described. And, finally, a database of about 2,000 literacy materials actually 
used by job incumbents was collected. The database is the largest of its kind in the countr>', and 
it is already being used in the writing of new NAEP items. 

The JLA process, as mentioned above, results in a job literacy description, which 
includes a representative collection of the most important materials required for competent 
performance of the job, and the relevant (linked) tasks. One additional step is critical to the 
process if generalizable assessment and training specifications are to be developed from the 
tasks and materials of the job literacy description. If assessment or training materials were 
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developed directly from the description, the product would be too specific; no allowance would 
be made for the transfer of learning which must occur if and when a literacy material were to be 
modified. However, if the type and level of the skills involved in performing the JLA tasks with 
the materials can be identified, training is more likely to generalize to the successful learning 
and completion of similar tasks based on those skills. A cost-effective and practical procedure is 
needed to examine the literacy materials and tasks and identify the skills involved. The results 
of the skills definition stage will form the basis for test and/or training design. The skills level 
of analysis is also an essential next step because it may identify skills that generalize within the 
relevant job cluster and within relevant career ladders. 

Once the skills analysis step has been completed, the new JLA methodology can be put 
to practical use in the design of different types of assessment and training for a broad variety of 
clients. 
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